FIESTA: A Sample-Balanced Multi-Program Workload Methodology

Workload construction methodologies for multiprogram experiments are more complicated than those for single-program experiments. Fixed-workload methodologies pre-select samples from each program and use these in every experiment. They enable direct comparisons between experiments, but may also yield runs of which significant portions are spent executing only the slowest program(s). Variable-workload methodologies eliminate this load imbalance by using the multi-program run to define the workload, normalizing performance to the performance of the resulting individual program regions. However, they make direct comparisons difficult and tend to produce workloads that over-estimate throughput and speedup. We propose a multi-program workload methodology called FIESTA which is based on the observation that there are two kinds of load imbalance. Sample imbalance is due to differences in standalone program running times. Schedule imbalance is due to asymmetric contention during multi-program execution. Sample imbalance is harmful because it dilutes multi-program behaviors. Schedule imbalance is a characteristic of concurrent execution that should be preserved and measured. Traditional fixed-workload methodologies admit both kinds of imbalance. Variable-workload methodologies eliminate both kinds of imbalance. FIESTA is a fixed-workload methodology that eliminates only sample imbalance. It does so by pre-selecting program regions for equal standalone running times rather than for equal